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stopped by this morning to drop off 
this se tons, with which they are begin= 
ning their investigation and study. I told that 

I knew from your comments to the Senate Select Com- 
mittee staffers that you have a clear conceptual view 

of what estimates mist and must not be and suggested that 
they interview you early in their canvassing. 
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CENTER FOR THE STUDY OF INTELLIGENCE 
6 August 1976 


Colleagues: 


Attached is a list of proposed questions/issues involving 
the Estimative Intelligence process. We would like to make it 
the basis for our discussion on 11 August. The list represents 
our initial view of the central and also some of the side issues 
we wish to cover in the Center project on Estimates. 


We would like your opinion on: 


--whether our listing is complete, or are there 
other key estimative issues that we've missed 
or have failed to focus on usefully? 


~-which are the most important issues? 


We would also appreciate your recollections of individual 
estimates that were of particular significance, perhaps because 
they were: 


--right on the mark or badly off it; 


--hit the policy machinery at the right spot with 
just the right timing, or were way off it; 


--particularly well or poorly written; well or 
poorly coordinated. 


Our purpose in secking this information is to select a 
sampling of past estimates for study to see if we can gain 
any insights about the process as a whole that can be usefully 
developed in the paper. 


Finally, your suggestions as to consumers, producers, 
and others whom we should interview in connection with this 
project would be appreciated. 


Looking forward to secing you, 
Wednesday, 11 August. 


Att, 
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6 August 1976 


THE ESTIMATIVE PROCESS AND PRODUCT 


Questions/Issues 


DEFINITIONS 
1. What is a "national estimate": 
--NIEs and SNIEs only, 


--or also other papers such as NIAMs, contribu- 
tions to NSSMs, 


--how about Alert Memoranda, Strategic Warning 
Staff publications, predictive papers produced 
by a single office (OPR) or agency, or the DCI 


2. What critcria must be met by a "national" estimate" 


--subject matter of interest to a "national-level" 
policy consumer?, 


--fully coordinated interagency paper?, 
--approval by the DCI? 
3. Who are the '"national-level" policy consumers? 


--only the President and the National Security 
Council members? 


--other Executive Agency and Department heads 
as well? 


--lower levels of the Executive Branch? how low? 
--the Congress? 
--are Intelligence officers also "consumers?" 
3a. How important is the estimative function as 


an element. of the finished intelligence process 
today? 
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--most important? 
--backscat to other products today? 
Il. USE/PURPOSE 
4. How are estimates used? 
~-decision making, planning, background? 
~-as guidance for other intelligence analysis? 


~-aS an aid to planning intelligence collection, 
analysis and resource allocation? 


5. What should be the purpose, or purposes, of estimates, 

and which are most important? 

--to support top-level national security policy- 
making only?, mainly? 


~-to really "estimate," or instead to set out the 
forces at work and alternative possible develop- 
ments, leaving the final judgments or estimating 
to the consumer? 


--to concentrate mainly on controversial subjects: 


~~those on which evidence, analysis and judg- 
ments differ, 


~"on matters in which intelligence appears to 
run counter to the generally accepted view, 


-~on the uncertain? 


~-to serve as a medium for systematic presentation 
of a variety of evidence and for illuminating 
conflicting interpretations and judgments for a 
wide range of consumers (policy and intelligence 
people)? r 


~-to provide an educational service, particularly 
for non-specialists in the subject matter of 
the estimate? 

~-is there a limit to the subject scope of estimates? 
~-stop short of cstimating outcomes in food, 


population development, etc. Stop estimating 
coup prospects and other similar, tricky subjects. 
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TIL. QUALITY OF ESTIMATES 
6. Are estimates relevant - do they mect the users needs? 


--do they cover the important subjects, address the 
right guestions? 


--are they too long-range, too short-range or near- 
Sighted? 


--too remote from "real" problems and concerns? 


--do they suffer from a lack or awareness of 
policy information, options or plans? 


~-what percentage are "nice to have" rather than 
needed? 


--how many are very valuable, useful, unnecessary? 
7. Are estimates "predictive" enough: 

--too fuzzy and overly qualified 

-~-too speculative 


~-contain too much/too little factual or background 
material 


--present analysis adequate to inspire confidence 
in judgments and support the conclusions 


--lacking in alternative interpretations 


--should an estimate cver say "we cannot come up 
With a judgment yet?" 


8. Are estimates too late to be of use? 
--usually, often, sometimes? 
Why are late estimates late? 
--does the request come too late? 
--are the cstimates too long? 
--is the preparation too slow?, why? 
~-overcoordination? 


--overload on analysts? 
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9. Is there a tendency in estimates to neglect overt 
information in favor of seerct intelligence? If, 
so docs this adversely affect the quality of 
estimates? 


IV. FORM AND PROCESS 
10. Are classification and controls a problem: 


--do estimates suffer from the omission of highly 
sensitive information (intclligence, policy and 
covert action, if any)? I£ so, would more 
flexible and limited distribution practices 
permit inclusion of such information? 


--are estimates too highly classified/controlled 
to be useable? If so, would sanitized versions 
or separate, higher classified annexes be 
desirable? 


11. Should estimates follow a more or less uniform pattern 
in content, structure and format? 


--Or should they be quite different for different 
types of subject matter and situations (political, 
economic, military, technical, long or short range, 
planning or crisis management, etc.)? 

--Should estimates be highly "tailored" for a specific 
consumer and distributed on a limited basis only to 
those requesting them? 

--Would and should such tailoring permit inclusion of 
specific policy options under consideration, thus 
making the estimate more relevant, useful? 

--Should we produce different versions of the same 
estimates for different recipients because of 
classification level, sensitivity, intended use, 
etc.? Would this improve their utility? 

12. How should estimates be initiated? 

--by the consumers, 

--the producers, 

--both? 


--at what level, 
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--with some machinery to accept or reject requests 
and set prioritics? 


--is there a need for a regular reexamination and 
updating of most cstimates? 


Are analytical resources adequate for the estimating 
task? 


-~is sufficient analytical effort devoted to estimating? 


--do competing tasks such as current intelligence 
drain off and interfere with the estimating 
function? 


--can the same analyst produce effective estimates 
if he has other analytical responsibilities? 


--do both estimating and current reporting benefit 
from being done by the same individual? 


--would more analysts with narrower fields and less 
workload pressure improve matters? 


--is the quality of analysis and analysts satisfactory? 


Are analytical methods used in estimating satisfactory? 
Are there "new" analytical methods which should be 
used, or used more? Identify, and give examples of 
estimates which could have profited from methodological 
innovation. 


Is there an adequately multidisciplinary approach to 
estimating? Is the interdisciplinary analyst or the 
interdisciplinary team the better approach? Other 
suggestions? 


Should there be an established procedure for soliciting 
for inclusion in estimates the views of those "on the 
scene'' (the Ambassador, country team, military repre- 
sentative, negotiator)? 


Do estimates adequately reflect the perceptions of 
“the other side" when appropriate? Or are estimates 
flawed by a tendency to perceive and judge the 
motives, actions and reactions of others by our 

own value system and culture? 


low are conflicting analyses and judgments best handled? 


-~-dissenting footnote? 


Approved For Release 2005/04/18 : CIA-RDP81B00493R000100010016-5 


LO 


20 4 


ais 


oes 


(oe 


Approved F&wRelease 2005/04/18 : CIA-RDP81 B0049@R0001 00010016-5 


--altcernative asscssments in one estimate? 


--separate estimates from competing analytical 
CCnters? 


Is there a problem of conflicting estimates appearing 
in different intelligence publications from different 
agencies at different times on the same question? 


Examples? Should there be more formal coordination? 
Or a "Clearinghouse" to alert the consumer to this 
situation on specific papers? 


Should estimating be left to the cxperts/specialists 
exclusively? Or is there a useful role for the 
generalist? If generalists can contributc, how is 
this best done, 


--ad hoc by individuals or teams? 
--by an institutionalized collegium? 


Does the value of a collegium of generalists (and 
specialists) as a separate forum for the discussion 
and consideration of estimates justify maintaining 
such an institution? If so, should it be “in house," 
interagency, outsiders, advisory or responsible for 
the final estimate? 


Are estimates too “inbred?" Should there be more 
participation by non-intelligence people: policy- 
makers, outsiders, contractors, academics? How 
should they fit into the process? 


Should estimates be routinely reviewed and critiqued 
by outside specialists who do not have access to intel- 
ligence data? Would such a process be a safeguard 
against giving undue weight to intelligence to the 
neglect of overt data and outside judgment? Would 

it counteract the "conventional wisdom" of the 

insider or help insulate cstimates from current 

policy bias? 


--can outsiders be drawn cffectively into the 
estimative process?  Ilow? 


~-is there a danger of a conflict of interest ., 
if non-governmental consultants were brought in? 
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Should the devil's advocate be institutionalized? 
If so, how? 


--by an outside review board? 


--an in-house group exclusively devoted to this 
role? Should the views of the devil's advocate 
be included in the estimate, appended, or issucd 
separately? 


Should there be a “post mortem" of every estimate, 
designed to judge how right or wrong it was, how 
relevant, useful; should there be an attempt to 
determine the causes for errors in estimates and 
ways to avoid repeating them? How should the 


“ process be conducted? 


Would estimates have more impact and be more uscful 
to the consumer if they were presented differently: 


--by briefings 
--with more graphics 
--in other ways? 


How important is consumer fcedback and how valid are 
producers! complaints about the lack of feedback? 


--should there be some kind of consumers committee 
charged with reviewing and guiding the estimative 
process? 


Does the search for policy relevance and consumer- 
producer interaction threaten the objectivity of 
estimates through too close association of intelli- 
gence analysts with policy and policy makers? 


Should more estimates be undertaken which are directly 


related to current or contemplated policy decisions 
and directions? Should these estimates deliberately 
test and question the validity of the assumptions on 
which these policies are based? Should the estimator 
in this situation play the devil's advocate role for 
the policy maker? 


What is the value and proper place of the "net 
assessment?" Should estimates include net assess- 
ments? Should net assessments be handled scparately? 
Or should the net assessment incorporate or replace 
the estimate? 
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31. Are there other questions or problems which should 
be explored? 


32. In view of all of the above, what are the most 


important steps which could be taken to improve 
intelligence estimates and the estimating process? 
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